Heuristic and Formal Methods in Automatic Program Debugging

I . I n t r o d u c t i o n TALUS acts as the domain expert of an intelligent tu tor ing system to teach LISP. A complete intelligent tutor ing system would include a student model, a dialog manager, courseware, and additional domain expertise. Input to TALUS consists of one or more student functions intended to solve an assigned task. Typical tasks include REVERSE, MEMBER, U N I O N , I N T E R S E C T I O N , F A C T O R I A L , and F L A T T E N . Outpu t consists of the debugged student functions and bug annotations. Whi le TALUS wi l l not recognize unanticipated algorithms, unanticipated implementations are permit ted. Associated w i th each task are representations of algorithms that solve the task. Heuristics match the student's buggy solution w i th the algori thm most similar to i t . Formal methods detect bugs in the student's functions by comparing them wi th This work was sponsored in part by NSF CER grant MCS-8122039 and Army Research Office grant DAAG29-84K-0060. stored functions. Heuristic methods suggest minimal alterations to the student's functions to remove the bugs; formal methods accept or reject the proposed alterations. I I . P r e v i o u s A p p r o a c h e s t o A u t o m a t i c P r o g r a m D e b u g g i n g Heuristic approaches to automatic program debugging parse student programs into an abstract representation and match that representation against either stored plan templates f rom a l ibrary [John84; Solo82], or a model program [Adam80]. Bugs appear as unaccountable differences between stored correct plans and the parsed program. Plan transforms increase the range of programs that can be accepted by statically representing common implementation variants. However, some correct student implementations wi l l require inductive proofs or unanticipated transforms to establish their equivalence to stored plans. These implementations are rejected as buggy or unanalyzable. A formal approach to automatic program debugging, by Katz and Manna [Katz76], extends the logical analysis of programs to include program incorrectness and a means of correcting incorrect programs. Program statements are related to synthesized inductive invariants. When these invariants are insufficient to establish a proof of correctness, program statements are altered so the necessary inductive invariants are derived. Synthesizing these inductive invariants and determining what program statements to alter is di f f icul t ; no implementation of their design exists to date. Shapiro [Shap82] traces the execution of pure PROLOG programs to isolate the presence of bugs in procedures whose traces are incorrect. The user supplies informat ion about examples to t ry , the correctness of program traces and violations of well founded relationships. Bugs are corrected by synthesizing correct clauses or by searching among perturbations of buggy clauses. This method can be applied to other functional programming languages, but